Machine Learning for Semi-Automated Gameplay Analysis

Compelling gameplay requires constant testing and refinement during the development process, amounting to a considerable investment in time and energy. This article presents an approach to gameplay analysis intended to support and augment the work of game designers, collecting and summarizing gameplay information from the game engine so designers can quickly evaluate the behaviour to make decisions. Using readily available machine learning technologies, a reusable tool has been constructed that can repeatedly choose scenarios to examine, run them through the game engine, and then construct concise and informative summaries of the engine's behaviour for designers. Based on the past scenarios, new scenarios are intelligently chosen to verify uncertain conclusions and refine the analysis. Game designers can examine the summaries produced by the analyzer, typically with a secondary visualization tool, providing the essential human judgement on what constitutes reasonable and entertaining behaviour. The inevitable role of the designer is why we use the term `semi-automated'. The analysis tool, SAGA-ML (Semi-Automated Gampeplay Analysis by Machine Learning), is based on machine learning research known as 'active learning', and has been used to evaluate Electronic Arts' FIFA'99 soccer game, uncovering some interesting anomalies in gameplay. With only minor changes, the tool was interfaced to FIFA 2004, and plays an active, in-house role in the development and testing of the FIFA series. While designed and developed in this context, the analysis tool is general purpose, requiring only a thin interface layer to be written to connect to different game engines. For the specific case of FIFA, a visualization tool, SoccerViz, has also been developed. SAGA-ML and SoccerViz were designed and developed by the University of Alberta GAMES group in cooperation with Electronic Arts. While presentation aspects like graphics and sound are important to a successful commercial game, it is likewise important that the gameplay, the non-presentational behaviour of the game, is engaging to the player. Considerable effort is invested in testing and refining gameplay throughout the development process. We present an overall view of the gameplay management problem and, more concretely, our recent research on the gameplay analysis part of this task. This consists of an active learning methodology, implemented in software tools, for largely automating the analysis of game behaviour in order to augment the abilities of game designers. The SAGA-ML (semi-automated gameplay analysis by machine learning) system is demonstrated in a real commercial context, Electronic Arts' FIFA'99 Soccer title, where it has identified exploitable weaknesses in the game that allow easy scoring by players.